Software Assurance Measurement – State of the Practice

iv 1 Background and Assumptions 1 1.1 The Importance and Challenges of Software Assurance Measurement 1 1.2 Why Security Is a Concern 1 1.3 Purpose of This Report 3 1.4 Intended Audience 3 1.5 Scope 4 1.6 Constraints 5 2 Definition of Terms 6 2.1 What Is Software Assurance? 6 2.2 What Is Software Assurance Measurement? 6 2.3 Measurement Domains 7 2.4 Critical Programming Errors 8 3 Measurement and Metrics 10 3.1 Definitions of Terms 10 3.2 The Metric Development Process 11 3.3 Functional and Structural Attributes of Correctness 11 3.4 Methods and Models 12 3.5 Measurement Management 13 3.6 Managing Through Measurement Baselines 14 3.7 Measurement Data Acquisition, Retention, and Use 17 4 Standard Assurance Measurement Methodologies 18 4.1 Measurement Processes 18 4.2 Assessment Technologies 18 4.3 Standard Assurance Environments: The Object Management Group Software Assurance (SwA) Ecosystem 19 4.4 Standard Assurance Environments: Common Weakness Enumeration (CWE) and Common Attack Pattern Enumeration and Classification (CAPEC) 20 4.5 Standard Assurance Environments: Consortium for IT Software Quality Characteristics Project 21 4.6 Software Productivity Research 22 5 Current Relevant Software Assurance Measures 23 5.1 Common Assurance Metrics 23 5.2 Direct and Indirect Measures of Software Performance 23 5.3 The Other Side of the Equation: Problems with Measures of Software Size 24 6 Summary and Conclusions 26 6.1 Business Realities Versus Due Care 27 6.2 Formal Measurement – Better Assurance 27 Appendix A: Common Measures by Life-Cycle Area 28 Common Productivity Measures 28 Requirements Measures 28 Design Measures 29 Code Measures 30